Flexible Multimodal Human-machi Environment

This article describes a prototype system for multimodal human-machine interaction in mobile environments that is being developed within the German national SmartKom research project. The system is novel in that it a) aims at combining the flexibility of state-of-the-art handheld digital devices with the computing power of standard PC machinery, and b) connects two substantially different mobile environments, namely pedestrian and car driver. Motivated by the specific safety and privacy considerations and requirements in both environments, we present a framework for flexible modality control. A characteristic feature of our framework is the insight that both user and system may independently and asynchronously initiate a modality transition. We conclude with a brief discussion of further issues and research questions.